TITLE OF THE INVENTION 
3D IMAGE ACQUISITION APPARATUS AND 3D IMAGE ACQUISITION 
METHOD 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This application is based upon and claims the 
benefit of priority from the prior Japanese Patent 
Application No. 2000-387428, filed December 20, 2000, 
the entire contents of which are incorporated herein by 
reference . 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a 3D (Three 
Dimensional) image acquisition method for a technique 
of measuring a 3D shape of an object in a non-contact 
state, and a 3D image acquisition apparatus for 
carrying out this method. 

2. Description of the Related Art 
Conventional 3D object shape measuring methods 

applied to 3D image acquisition apparatuses as 3D 
shape measuring techniques are generally classified 
into two types. One is a method in which the time of 
propagation of light is measured, and the other is 
a method making use of a "principle of triangulation" . 
The former is a theoretically ideal method because 
there is no dead angle, but it has problems with 
measured time and precision. Thus, the latter using 
"principle of triangulation" is mainly used. 



Methods using the principle of triangulation are 
classified into an active method and a passive method 
called "passive stereo method". In the passive stereo 
method, features in the images obtained by two cameras 
situated at different locations are made to correspond. 
On the basis of a result of the correspondence and a 
pre-measured positional relationship between two 
cameras, a distance up to an object is found by the 
principle of triangulation. The passive stereo method, 
however, has drawbacks that the correspondence of 
features in the images is difficult and the shape of an 
object without texture cannot be measured. 

As an example of the active method using the 
principle of triangulation, there is known a "light 
projection method". In this method, one of the two 
cameras, is replaced with a light source, and an image 
of the light source at the surface of the object is 
observed by the camera situated at the other point and 
the shape of the object is measured. The "light 
projection method" is also classified into "spot light 
projection method", "slit light projection method" and 
"pattern light projection method". 

In the spot light projection method, only a point 
of an object can be measured by a single image input. 
In the slit light projection method, one line of 
an object can be measured by a single image input. 
However, in order to measure the shape of the entire 



area of the object, image input needs to be repeated 
a number of times by scanning projection light, and 
a great deal of time is required. 

On the other hand, in the pattern light projection 
method, a two-dimensional (2D) pattern such as a stripe 
pattern or a matrix pattern is projected. This method 
is advantageous in that only one input of a pattern 
projection image needs to be performed. 

The pattern light projection method is also called 
"spatial encoding method". The "spatial encoding 
method" is classified into "pattern shape encoding" and 
"gradation encoding". As the pattern shape encoding, 
a method using a distribution of opening width of 
a slit and a method using an M-series encoding have 
been proposed, but these are less practical because of 
problems with measurement density and stability of 
measurement . 

Meanwhile, the "gradation encoding" is classed 
into "gray scale encoding" and "color tone encoding". 
However, either method has a drawback that the number 
of recognizable gradations is small. As regards the 
gray scale encoding, the maximum number of recognizable 
gradations is normally 5. As regards the color tone 
encoding, a greater number of gradations are not 
usable, except the cases of white and low-saturation 
colors . 

To solve these problems, a method has been 
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proposed wherein one code is assigned to a combination 
a plurality of colors or gray levels. 

Prior-art matrix pattern encoding and stripe 
pattern encoding will now be described. 

As regards the matrix pattern, Patent 
No. 2,565,885, for instance, discloses a "spatial 
pattern encoding method" . This method employs a 
multi-pattern matrix plate pattern, which has gray 
scale areas of three or more values, color areas of 
three or more colors, or three or more gradation areas 
of a combination of gray scales and colors. In this 
method, at least three kinds of gradation areas adjoin 
at an intersection of boundary lines of gradation 
areas. Major codes corresponding to the kinds and 
order of gradations are assigned to intersections of 
a projection image produced by projecting the pattern 
on the object. Then, the major codes, or combinational 
codes of major code at the intersection and major codes 
of intersections around this intersection, are given as 
characteristic codes for intersection recognition. 

FIG. 9 shows an example of the matrix pattern 
using three-value gradations in the above-described 
prior-art spatial pattern encoding method. White is 
represented by "2", black is by "0", and gray is 
by "1". FIG. 10 is a block flow chart illustrating 
the process by the spatial pattern encoding method. 

In step ST1001, recognition thresholds for 



three-value conversion are found. In step ST1002, 
three-value conversion for each pixel is effected, 
like "0", "1" or "2". 

In step ST1003, node selection is effected to 
extract, as a node, the corner or edge intersection 
of each matrix plate area of the multi-value matrix 
pattern. 

In step ST1004, a decoding process is performed to 
provide one of a plurality of codes as a major code in 
accordance with a gradation distribution surrounding 
the extracted node. 

In step ST1005, a result of code recognition and 
code numbering is recorded on a table (memory) . 

This method is advantageous in that it provides a 
pattern structure which can produce many characteristic 
codes by only single projection of the pattern with at 
least three gradations. 

In order to derive encoded information from 
a received pattern and derive 3D information, it is 
important to exactly read a luminance variation and to 
detect as exactly as possible boundaries of one block 
in the matrix. A graph shown below the matrix pattern 
in FIG. 9 indicates gradations of the second line from 
above in the matrix pattern. The inclination of a 
graph curve increases near inflection points (i.e. two 
occurrences of (1) in (1)(2)(1) in FIG. 9), which are 
neither local maximum points or local minimum points, 
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due to diffusion of projected light. Consequently, 
extraction of a matrix structure (division of blocks 
in the matrix) becomes disadvantageously unstable. 
In short, the degree of deviation to the left and right 
5 of an inflection point increases, and the reliability 

lowers . 

FIG. 11 shows prior art in the case where 
gradations of three colors, R (Red) , G (Green) and B 
(Blue) , are used in the spatial pattern encoding 

10 method. Assume that "0" is assigned to R, "1" to G, 

and "2" to B. 

A graph below the matrix pattern in FIG. 11 
indicates luminance values of R, G and B of the second 
line from above in the matrix pattern. For example, in 

15 the case of an object (a high-saturation red object) 

having such surface reflectance characteristics that 
reflection signals of a G component and a B component 
are substantially zero while a reflection signal of an 
R component is large, a signal of a local maximum is 

2 0 obtained at the luminance value of R in the second line 

from above in the pattern of FIG. 11. By contrast, the 
luminance values of G and B have nearly noise levels. 
In this situation, decoding is impossible. Similarly, 
decoding is very difficult for an object having such 

25 surface reflectance characteristics that even one of 

reflection signals of R, G and B is substantially zero. 
In order words, decoding is successfully carried out 
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for an object surface of a white-based color or a low- 
saturation color. However, for an object having such 
surface reflectance characteristics that even one of 
reflection signals of R, G and B components may take 
5 a nearly zero value, determination based on a color 

obtained from a combination of R, G and B is difficult. 

Prior art of a stripe pattern, which is disclosed 
in, e.g. the Journal of the Institute of Electronics 
and Communication Engineers (Vol. J61-D, No. 6, 

10 pp. 411-418, 1978), will now be described. FIG. 12 

shows an example of a stripe pattern encoded by colors 
of R, G and B. This example differs from the example 
of FIG. 11, not only because the former is a stripe and 
the latter is a matrix, but also because black areas 

15 (hatched in FIG. 12) are present among R, G and B. 

The presence of black areas reduces the possibility of 
mixing of colors due to diffusion of light. 

However, in this prior-art example, too, decoding 
is difficult for an object having such surface 

20 reflectance characteristics that even one of reflection 

signals of R, G and B components may take a nearly zero 
value . 

The present invention has been made to solve the 
problems of the above-described prior art. The object 
25 of this invention is to provide a 3D image acquisition 

method and a 3D image acquisition apparatus based on 
spatial pattern encoding, wherein stripe boundary 



extraction from a stripe pattern projection image 
and block boundary extraction from a matrix pattern 
projection image can exactly be performed, and exact 
decoding is performed from an encoded projection 
image of an object even where the object is not of 
a white-based color or a low-saturation color. 

BRIEF SUMMARY OF THE INVENTION 

In order to solve the above problems and achieve 
the object, the present invention provides the 
following means. Specifically, in the 3D image 
acquisition method of this invention, spatial pattern 
encoding is used to measure the shape of a 3D object 
in a non-contact state. A pattern is projected, and 
the correspondency between a reception-light pattern 
and a projection-light pattern is found. Thereby, 
depth information of an object is acquired. The 3D 
image acquisition apparatus of this invention is 
provided to carry out this method. 

According to a first aspect of the invention, 
there is provided a 3D image acquisition apparatus 
comprising: a pattern projection section which projects 
a predetermined pattern on an object; an imaging 
section which is disposed at a predetermined distance 
from the pattern projection section and images the 
object on which the predetermined pattern has been 
projected; and a depth calculation section which 
detects the projection pattern projected on the object 



on the basis of an image acquired by the imaging 
section, collates the detected projection pattern and 
the predetermined pattern projected by the pattern 
projection section, and calculates a depth of 
respective parts of the object on the basis of the 
correspondency of the collation, characterized in that 
the predetermined pattern projected by the pattern 
projection section is formed by alternately arranging 
areas with local maximum luminance values and areas 
with local minimum luminance values. 

According to a second aspect of the invention, 
there is provided a 3D image acquisition apparatus 
comprising: a pattern projection section which projects 
a predetermined pattern on an object; an imaging 
section which is disposed at a predetermined distance 
from the pattern projection section and images the 
object on which the predetermined pattern has been 
projected; and a depth calculation section which 
detects an image pattern on the basis of an image 
acquired by the imaging section, collates the detected 
image pattern and the predetermined pattern projected 
by the pattern projection section, and calculates 
a depth of respective parts of the object on the basis 
of the correspondency of the collation, characterized 
in that the predetermined pattern projected by the 
pattern projection section is formed by combining 
patterns of a plurality of color components, and the 



patterns of the plurality of color components are 
formed by alternately arranging areas with local 
maximum luminance values and areas with local minimum 
luminance values . 

In this 3D image acquisition apparatus, in 
the patterns of the plurality of color components, 
positions of the areas with local maximum luminance 
values and positions of the areas with local minimum 
luminance values are aligned. 

In this 3D image acquisition apparatus, in 
the patterns of the plurality of color components, 
positions of the areas with local maximum luminance 
values and positions of the areas with local minimum 
luminance values are shifted. 

In the above 3D image acquisition apparatus, 
the color components are RGB components. 

In this 3D image acquisition apparatus, the 
projected predetermined pattern is a stripe pattern or 
a matrix pattern. 

According to a third aspect of the invention, 
there is provided a 3D image acquisition method 
comprising: a step of projecting a predetermined 
pattern on an object; a step of imaging the object on 
which the predetermined pattern has been projected, at 
a position at a predetermined distance from a position 
where the predetermined pattern has been projected; and 
a step of detecting a pattern on the basis of an image 
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acquired by the imaging step, collating the detected 
pattern and the pattern projected by the pattern 
projection step, and calculating a depth of respective 
parts of the object on the basis of the correspondency 
5 of the collation, characterized in that the predeter- 

mined pattern projected by the pattern projection step 
is formed by alternately arranging areas with local 
maximum luminance values and areas with local minimum 
luminance values. 

10 According to a fourth aspect of the invention, 

there is provided a 3D image acquisition method 
comprising: a step of projecting a predetermined 
pattern on an object; a step of imaging the object on 
which the predetermined pattern has been projected, at 

15 a position at a predetermined distance from a position 

where the predetermined pattern has been projected; and 
a step of detecting a pattern on the basis of an image 
acquired by the imaging step, collating the detected 
pattern and the pattern projected by the pattern 

20 projection step, and calculating a depth of respective 

parts of the object on the basis of the correspondency 
of the collation, characterized in that the predeter- 
mined pattern projected by the pattern projection step 
is formed by combining patterns of a plurality of color 

25 components, and the patterns of the plurality of color 

components are formed by alternately arranging areas 
with local maximum luminance values and areas with 
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local minimum luminance values. 

Additional objects and advantages of the invention 
will be set forth in the description which follows, and 
in part will be obvious from the description, or may 
be learned by practice of the invention. The objects 
and advantages of the invention may be realized and 
obtained by means of the instrumentalities and 
combinations particularly pointed out hereinafter. 
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 

The accompanying drawings, which are incorporated 
in and constitute a part of the specification, 
illustrate embodiments of the invention, and together 
with the general description given above and the 
detailed description of the embodiments given below, 
serve to explain the principles of the invention. 

FIG. 1 illustrates a light pattern structure when 
pattern light is projected in a 3D image acguisition 
method according to a first embodiment of the present 
invention; 

FIG. 2 schematically shows the structure of a 3D 
image acguisition apparatus according to the first 
embodiment; 

FIG. 3 is a block flow chart illustrating the 
process in the 3D image acquisition method according to 
the first embodiment; 

FIG. 4 schematically shows the structure of a 3D 
image acquisition apparatus according to a second 



embodiment of the invention; 

FIG. 5 illustrates a light pattern structure when 
pattern light is projected in a 3D image acquisition 
method according to a third embodiment of the 
invention; 

FIG. 6 is a block flow chart illustrating the 
process in the 3D image acquisition method according to 
the third embodiment; 

FIG. 7 illustrates a light pattern structure when 
pattern light is projected in a 3D image acquisition 
method according to a fourth embodiment of the 
invention; 

FIG. 8 illustrates a light pattern structure when 
pattern light is projected in a 3D image acquisition 
method according to a fifth embodiment of the 
invention; 

FIG. 9 shows an example of a matrix pattern using 
three-value gradations in a prior-art spatial pattern 
encoding method; 

FIG. 10 is a block flow chart illustrating the 
process in the prior-art spatial pattern encoding 
method; 

FIG. 11 shows an encoded light pattern structure 
in prior-art matrix pattern light projection; and 

FIG. 12 shows a light pattern structure in 
prior-art matrix pattern light projection. 
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DETAILED DESCRIPTION OF THE INVENTION 
Embodiments of the present invention will now be 
described with reference to FIGS. 1-8. 
(First Embodiment) 
5 A first embodiment of the invention will now be 

described in detail with reference to FIGS. 1-3. 

In order to describe a pattern light projection 
method for spatial pattern encoding according to the 
first embodiment, FIG. 1 shows a four-gradation stripe 
10 pattern by way of example. This pattern is a predeter- 

mined light pattern which is projected on an object 
when a 3D image of the object is to be acquired, for 
example, in order to measure the shape of the object. 
This predetermined pattern comprises a plurality 
15 of areas (stripes in FIG. 1) each having a substan- 

tially constant luminance value. The luminance 
(gradation) of each area belongs to any one of 
predetermined luminances (gradations) . In FIG. 1, 
each stripe belongs to any one of four gradations (0, 
20 1, 2, 3) . In this example, pattern areas having local 

maximum luminance values and local minimum luminance 
values are alternately arranged. 

Each area is arranged such that if one of its 
adjacent areas has a higher luminance (gradation) than 
25 itself, the other adjacent area also has a higher 

luminance (gradation) than itself. In addition, each 
area is arranged such that if one of its adjacent 



areas has a lower luminance (gradation) than itself, 
the other adjacent area also has a lower luminance 
(gradation) than itself. Both adjacent areas have 
different luminances (gradations) . 

In the encoding of respective stripe patterns, 
as shown in the graph of FIG. 1, local maximum values 
and local minimum values are alternately arranged. 
Local maximum values of gradations 1, 2 and 3 are 
expressed by (1), (2) and (3), and local minimum values 
of gradations 0, 1 and 2 are expressed by (0), (-1) and 
(-2) . Using these expressions, a description will be 
given of the spatial pattern encoding method in which 
the same code does not appear twice in the arrangement 
of the stripes. In this case, the order of stripe 
patterns are determined under conditions that the 
gradation of a location minimum is lower than the 
gradations of right and left local maximums, 
odd-number-th stripes have local maximum values, and 
even-number-th stripes have local minimum values. 

In the example of FIG. 1, if four numerals 
arranged in the order of a local maximum, a local 
minimum, a local maximum and a local minimum are 
encoded into one code, two codes, i.e. [(1) (0) (2) (-1)] 
and [(2) (-1) (3) (-2)], are obtained. In this case, the 
two successive codes share numerals (2) and (-1), which 
are the right-hand two numerals of the left-hand code 
and are the left-hand two numerals of the right-hand 



code. If codes are arranged to share the same 
numerals, arrangements of (2m+2) stripes are obtained 
when the number of codes is m. When the number of 
gradations is 4 and the code length is 4, the number 
of codes, m, is 30. Accordingly, in the case where 
information of 4 gradations is combined with local 
maximum/minimum information, if encoding is effected 
by the arrangement of four stripes, 62 stripes are 
obtained. 

The spatial pattern encoding process according 
to the present invention will now be described with 
reference to a flow chart of FIG. 3. In step ST301, 
stripes at positions of local maximums and local 
minimums are detected on the basis of a pattern 
projection image. These stripes can be detected by 
extracting edge portions of stripes through a "Sobel" 
filter or the like and determining whether an area 
interposed between the detected edge portions has 
a local maximum or a local minimum. 

In the next step ST302, the gradation of the area 
interposed between the edge portions is determined. 

In step ST303, a decoding process is performed 
by numerically converting the information of local 
maximum/minimum of the area between edge portions and 
the gradation information. In step ST304, the obtained 
numerical information is recorded on a table in a 
memory. The numerical information is collated with 



prestored information in a pattern memory 101 (to be 
described later) . 

In the case of four gradations, a code having the 
coincident order of arrangement of four numerals is 
found. If there are two or more coincident codes, 
a code having a greatest length of a successive 
coincident portion is selected as one with highest 
reliability. In this way, the correspondency of 1 to 
1 (at most) is determined between each stripe of 
a projection-light pattern and each stripe of a 
reception-light pattern, and depth-dimensional 
information of the object to be measured is calculated 
from the well-known "principle of triangulation" . 

A description will now be given of a 3D image 
acquisition apparatus for acquiring a 3D image by using 
a pattern obtained by the above-described spatial 
pattern encoding method. 

FIG. 2 schematically shows an example of the 
structure of a 3D image acquisition apparatus according 
to the first embodiment. This 3D image acquisition 
apparatus projects a predetermined pattern on an object 
to be measured, acquires an image of the object, and 
collating an obtained detection pattern and the 
projected pattern, thereby measuring depth dimensions 
of respective parts of the object. This apparatus 
comprises the following elements. 

A pattern memory 101 that prestores a 



predetermined pattern of projection light is connected 
to a pattern projection section 102 for projecting the 
light of the predetermined pattern on an object to be 
measured (not shown) . This predetermined pattern is 
projected from the pattern projection section 102 on 
the object in the state in which areas with local 
maximum luminance values and local minimum luminance 
values are alternately arranged, for example, as shown 
in FIG. 2. 

Imaging means for imaging the object, on which the 
predetermined pattern has been projected, is disposed 
at a predetermined distance from the pattern projection 
section 102. The imaging means comprises a lens 103 
for collecting reflection light from the object, and a 
light reception section 104 for receiving the collected 
reflection light. 

An image memory 105 that stores information of the 
light received by the light reception section 104 of 
the imaging means is connected to the imaging means. 
The image memory 105 and pattern memory 101 are 
connected so that a pattern can be detected on the 
basis of an image acquired by the imaging means (the 
imaging section including the light collection lens 103 
and light reception section 104) . A depth calculation 
section 106 is provided as depth calculation means 
for collating the detected pattern with the pattern 
projected by the pattern projection section 102, 



and calculating the depth dimension of the respective 
parts of the object on the basis of the correspondency 
between the two collated patterns. 

A 2D image information generating section 107 for 
generating 2D image information on the basis of the 
information stored in the image memory 105 is connected 
to a 3D image generating section 108. The 3D image 
generating section 108 generates a 3D image on the 
basis of the 2D image information generated by the 2D 
image information generating section 107 and the depth 
dimension value calculated by the depth calculation 
section 106. 

The operation of the 3D image acquisition 
apparatus will now be described. 

Pattern encoding information is prestored in the 
pattern memory 101. To start with, in order to acquire 
an image without a pattern, the light collection lens 
103 collects reflection light from the object, and the 
light reception section 104 receives the collected 
reflection light. The information of the received 
light is stored in the image memory 105. The 2D image 
generating section 107 generates two-dimensional 
photographic information. Based on this information, 
surface reflectance characteristics of the object are 
obtained. The surface reflectance characteristics of 
the object obtained from the image with no pattern are 
used as a base (reference) for comparison of luminance, 
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when pattern information is extracted from the image 
acquired from the object on which the pattern light is 
projected. 

A predetermined pattern is projected on the 
5 object, and reflection light from the object is 

collected by the collection light 103. The collected 
light is received by the light reception section 104 
and then stored in the image memory 105. The depth 
,_. calculation section 106 compares and collates the 

5f 10 information stored in the pattern memory 101 and the 

i y pattern information stored in the image memory 105, 

z_ thereby determining the correspondency between the 

=p projection-light pattern and the reception-light 

H pattern. 

m 

H 15 The incidence angle is calculated on the basis of 

Q the information of the light reception point and the 

H 

focal distance, and the depth dimension is calculated 
by the principle of triangulation on the basis of the 
information of the distance between the light emission 

20 point and light reception point and the light emission 

angle. Since the optical parameters (e.g. focal 
distance) of the lens 103 and the size of the light 
reception section are known, the incidence angle is 
determined on the basis of the position (light 

25 reception position) on the acquired image. The 

distance between the light emission point and light 
reception point is determined by the structure of the 



apparatus. The light emission angle is determined by 
the pattern on the image. 

At last, the 3D image generating section 108 
generates a 3D image on the basis of the 2D image 
generated by the 2D image generating section 107 and 
the depth dimension value calculated by the depth 
calculation section 106. 

According to the above-described structure of the 
3D image acguisition apparatus of the first embodiment, 
the areas with local maximums and the areas with local 
minimums are alternately arranged on the pattern 
employed. Thus, the stripe image (stripe boundary) can 
easily be extracted. 

Moreover, each band of R, G and B is encoded based 
on the combination of local maximum/ minimum information 
and gradation information. The range of objects for 3D 
shape measurement is not limited to white-based color 
objects and low-saturation color objects and is 
increased. 
(Second Embodiment) 

A second embodiment of the present invention will 
now be described with reference to FIG. 4. The second 
embodiment is substantially similar to the first 
embodiment shown in FIG. 2 with respect to the spatial 
pattern encoding method. However, as shown in FIG. 4, 
the structure of the 3D image acquisition apparatus is 
different, as described below. The imaging means are 
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provided at a plurality of locations with different 
viewpoints, for example, on the right and left sides 
(two locations) of the pattern projection section 202. 
Accordingly, a pair of image memories are provided on 
5 the right and left sides. Moreover, means for 

determining the right-and-left correspondency is 
additionally provided. 

More specifically, the 3D image acquisition 
apparatus according to the second embodiment comprises 

10 a pattern memory 201 storing a predetermined 

projection-light pattern; a pattern projection section 
202 for projecting the pattern light on the object; 
light collection lenses 203R and 203L and light 
reception sections 204R and 204L constituting, e.g. 

15 right and left imaging means disposed at two locations; 

right and left image memories 205R and 205L; a corre- 
spondency determination section 206 for determining 
the correspondency between the projected pattern and 
detected pattern; a 2D image information generating 

20 section 207; a depth calculation section 208; and a 3D 

image generating section 209. 

The operation of the 3D image acquisition 
apparatus according to the second embodiment will now 
be described. 

25 Like the first embodiment, images without patterns 

are acquired in advance. These images acquired at two 
locations with different viewpoints are stored in the 



image memories 205R and 205L. 

If a predetermined pattern is projected on the 
object, the projected pattern is reflected by the 
object and collected by the collection lenses 203R and 
203L disposed at two different positions (L (left) and 
R (right) in this case) . The collected reflection 
light is received by the light reception sections 204R 
and 204L, respectively, and then stored in the image 
memories 205R and 205L. 

The correspondency determination section 20 6 
performs four kinds of collations between: (D the 
projection-light pattern and the pattern received 
by the left light reception section 204L; (2) the 
projection-light pattern and the right light reception 
section 204R; (3) the images without patterns received 
by the left light reception section 204L and the right 
light reception section 204R; and @ the images with 
patterns received by the left light reception section 
204L and the right light reception section 204R. This 
considerably reduces the difficulty in the "passive 
stereo method" in the collation with non-texture areas. 

In the first embodiment, the correspondency 
between the projection-light pattern and reception- 
light pattern is used as the basis for calculation of 
depth dimensions. In the second embodiment, the 
correspondency between the right and left images 
without patterns and the correspondency between the 



right and left images with patterns are also used, and 
the collation with higher reliability can be realized. 

At last, the 3D image generating section 209 
generates a 3D image on the basis of the 2D image 
information obtained by the 2D image information 
generating section 207 and the depth dimension value 
calculated by the depth calculation section 208. 

As has been described above, according to the 
structure of the 3D image acquisition apparatus of 
the second embodiment, pattern projection images are 
obtained by the imaging elements situated at locations 
of two or more viewpoints. Not only the correspondency 
between the projection-light pattern and reception- 
light pattern but also the correspondency between 
plural images is performed. Thus, more exact 
correspondency is achieved and the measurement 
precision and reliability are enhanced. 
(Third Embodiment) 

A third embodiment of the present invention will 
now be described with reference to FIGS. 5 and 6. 

The structure of the 3D image acquisition 
apparatus used in the first embodiment is substantially 
the same as that of the 3D image acquisition apparatus 
of the third embodiment. However, the spatial pattern 
encoding method is different, as shown in FIG. 5. 

FIG. 5 shows an example of a stripe pattern in 
which local maximums/minimums of a plurality of color 



signals are aligned. Gray scale values are used in 
the structure of the pattern adopted in the first 
embodiment. On the other hand, the pattern used in 
the third embodiment is a pattern of respective color 
components (R, G, B) , and areas with local maximum 
luminance values and areas with local minimum luminance 
values are aligned. As is shown in FIG. 5, in the 
color stripe pattern, local maximum positions and local 
minimum positions of RGB are aligned. 

R, G and B signals are individually encoded, and 
a decoding process is carried out by using R, G and B 
signals whose reflectances are not at noise level. 
If the reflectances of all R, G and B signals are not 
at noise level, a color decoding process is attempted, 
and color information is used in the decoding process. 
In addition, since local maximums and local minimums 
are alternately present, the extraction of a pattern 
image is advantageously facilitated. 

A processing operation relating to the 3D image 
acquisition apparatus according to the third embodiment 
will now be described. The flow chart of FIG. 6 
illustrates the process based on the 3D image 
acquisition method. 

In step ST601, image data without a pattern is 
input . 

In steps ST602R, ST602G and ST602B, the R, G and B 
components of the image data are determined, and the 



amount of projected R, G and B light returned from the 
object is checked. Thereby, the reflectance character- 
istics of the surface of the object are approximately- 
determined. At this time, if the R component of the 
reflection light is at noise level and near zero, the R 
component of the pattern projection image is also near 
zero in the near region. Thus, the data of the R 
component cannot be decoded. The same applies to the 
data of the G component and B component. 

The above matters are determined. If the data 
of the R component of R, G and B of the image, for 
example, is not at noise level, the data of the R 
component of the pattern projection image is used 
(ST603R) . At this time, in step ST604R, the local 
maximum/minimum of the R component data is detected. 
In subseguent step ST605R, the gradation of the local 
maximum/minimum area is found. Similarly, the data of 
the G component and B component is subjected to the 
same processing (ST603G, ST604G, ST605G; ST603B, 
ST604B, ST605B) . 

On the other hand, in steps ST602T to ST605T, 
the following process is performed. If the surface 
reflectance characteristics of R, G and B are all above 
the noise level, the colors of the projection-light 
pattern are estimated. The estimation of colors is 
carried out by dividing the RGB components of the 
pattern projection image by the RGB components of the 



image with no pattern and finding their ratio. If the 
surface reflectance characteristics of R, G and B are 
all higher than the noise level, the RGB data of the 
pattern projection image is used to detect local 
maximums/ mi nimums of the data and to determine the 
colors . 

In step ST606, a decoding process is performed 
by numerically converting the information of local 
maximum/minimum of the area to be processed and the 
gradation information. In step ST607, the obtained 
numerical information is recorded on the table in the 
memory. Then, the numerical information is collated 
with prestored information in the pattern memory 101. 
Based on the principle of triangulation, the depth 
information of the object is calculated. 

In the pattern used in the 3D image acquisition 
apparatus of the third embodiment, the areas with local 
maximum luminance values and the areas with local 
minimum luminance values of R, G and B are aligned. 
The decoding can be performed on the basis of not 
only the luminance values and local maximum/minimum 
information of R, G and B in the graph, but also the 
color information determined by the R, G and B values 
in each stripe. 

Even where two of the three R, G and B signals are 
not at the noise level, the two signals can be used for 
the decoding. 
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Since encoding for R, G and B is individually- 
performed, the decoding process can be performed by 
combining the individual encoding of R, G and B and 
the reliability for the decoding is enhanced. 
5 (Fourth Embodiment) 

FIG. 7 shows an example of a pattern structure 
for use in pattern light projection in a 3D image 
acquisition method according to a fourth embodiment of 
■ Ui the invention. 

"% 10 The structure of the 3D image acquisition 

% apparatus used in the fourth embodiment is substan- 

zi tially the same as that of the 3D image acquisition 

y; 

"I" apparatus of the third embodiment. However, the form 

^ of the pattern used in the spatial pattern encoding 

N ; 15 method is different. Specifically, as shown in FIG. 7, 

Q the pattern used in this embodiment is a color stripe 

H 

pattern in which local maximum/minimum positions of R, 
G and B signals are shifted. More specifically, the 
local maximum position of the R signal is displaced 

20 from the local maximum positions of the remaining G and 

B signals. In other words, in the stripe pattern of 
this embodiment, the local maximum luminance values and 
local minimum luminance values of each of the R, G and 
B component are alternately arranged, and the local 

25 maximum position of the R signal coincides with the 

local minimum positions of the remaining G and B 
signals . 



In the pattern used in the 3D image acquisition in 
the fourth embodiment, the area with a local maximum 
luminance differs from the area with a local minimum 
luminance, and the local maximum positions and local 
minimum positions of R, G and B are shifted. In more 
general terms, in the pattern used in this embodiment, 
the areas with local maximum luminance values and the 
areas with local minimum luminance values of each of 
the color components are alternately arranged, and the 
position of the area with the local maximum luminance 
(the position of the area with the local minimum 
luminance) of the pattern of at least one of the color 
components coincides with the position of the area with 
the local minimum luminance (the position of the area 
with the local maximum luminance) of the patterns of 
the other color components. 

The method using this pattern is advantageous 
in that if the reflectances of all R, G and B signals 
are not at the noise level, the chroma become higher 
than in the case of the third embodiment in which the 
local maximum/minimum positions of all R, G and B are 
aligned, and the color determination process becomes 
easier than in the third embodiment. 
(Fifth Embodiment) 

The structure of a 3D image acquisition apparatus 
used in the fifth embodiment is substantially the same 
as that of the 3D image acquisition apparatus of the 



first embodiment. However, the form of the pattern 
used in the spatial pattern encoding method is 
different, as will be described below. 

FIG. 8 shows a pattern used in the 3D image 
acquisition method according to the fifth embodiment of 
the invention. This pattern structure is characterized 
in that the pattern is a gradation matrix pattern using 
gray scale values and local maximums and local minimums 
are alternately arranged in a two-dimensional fashion. 

Since the local maximums and local minimums are 
alternately arranged in a two-dimensional fashion 
in the pattern used in the fifth embodiment, the 
extraction of a matrix image (boundaries of a block in 
a matrix) is advantageously facilitated, compared to 
the prior art. 

According to this stripe pattern, encoding is 
effected by using 2X2 blocks, instead of encoding 
using four successive numerals. Thus, the possibility 
of breaking of a code is advantageously reduced in 
the case of measuring a vertically elongated object or 
a square object. 
(Sixth Embodiment) 

Although not shown, a sixth embodiment of the 
invention employs a matrix pattern in which blocks 
of the matrix, which have local maximum and minimum 
luminance values, are aligned among R, G and B. 
The structure of the 3D image acquisition apparatus may 
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be substantially equivalent to that of the preceding 
embodiment . 

According to the sixth embodiment, since encoding 
for R, G and B is individually performed, the decoding 
process can be performed by combining the individual 
encoding of R, G and B and the reliability for the 
decoding is enhanced. 
(Seventh Embodiment) 

A seventh embodiment of the invention employs 
a matrix pattern (not shown) configured to have blocks 
of local maximum/minimum luminance values shifted among 
R, G and B. 

Similarly, the structure of the 3D image 
acquisition apparatus may be substantially equivalent 
to that of the preceding embodiment. 

According to the seventh embodiment, the use of 
this matrix pattern is advantageous in that if the 
reflectances of all R, G and B signals are not at the 
noise level, the chroma become higher than in the case 
where the blocks of local maximum/minimum luminance 
values of all R, G and B are aligned, and the color 
determination process becomes easier. 
(Modification) 

The present invention may be modified as follows. 
In the above-described embodiments, the luminance 
(gradation) pattern adopts four gradations. From the 
standpoint of wide variety, it is preferable to provide 



32 



at least three gradations. Besides, from the 
standpoint of easy recognition, it is preferable to 
provide six or less gradations, more preferably, five 
or less gradations. 

It is desirable that the entire pattern has the 
feature that areas with local maximum luminance and 
areas with local minimum luminance are alternately 
arranged. However, since corner portions are not 
actually used, there arises no practical problem if 
that part of the pattern, which is within the visual 
field of the imaging optical system, has this feature. 
Considering the fact that the distance between the 
pattern projection means and the imaging means is 
sufficiently less than the distance between the pattern 
projection means and the object, it should suffice if 
this feature is realized on that part of the projected 
pattern, which is projected within the range of the 
same solid angles as the imaging angles of the imaging 
means . 

In the case where the imaging angles are 10° 
in the vertical direction and 15° in the horizontal 
direction, there is no practical problem if that part 
of the projected pattern, which is projected within the 
range of 10° in the vertical direction and 15° in the 
horizontal direction, has the feature that areas with 
local maximum luminance and areas with local minimum 
luminance are alternately arranged. 
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Other modifications may be made without departing 
from the spirit of the invention. 

The embodiments of the invention have been 
described. The present specification includes the 
5 following inventions. 

(1) There is provided a spatial pattern encoding 
method wherein blocks or areas with local maximum and 
minimum luminance values are alternately arranged and 
y, a code is generated by combining local maximum/minimum 

== 10 information and gradation information. Since the local 

^ maximums and local minimums are alternately arranged, 

extraction of a stripe image (stripe boundary) can be 
4= made easy. 

H (2) There is provided a spatial pattern encoding 

M 15 method of item (1) wherein a stripe pattern or a matrix 

Q pattern is employed. Pattern projection images are 

H 

obtained by imaging elements situated at locations of 
two or more viewpoints. Not only the correspondency 
between a projection-light pattern and a reception- 

2 0 light pattern but also the correspondency between 

plural images is performed. Thus, more exact 
correspondency is achieved and the measurement 
precision and reliability are enhanced. 
(3) There is provided a spatial pattern encoding 

25 method of item (1) wherein encoding is performed in 

substantially independent wavelength bands of, e.g. R, 
G and B. Since encoding for R, G and B is individually 
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performed, the decoding process can be performed by 
combining the individual encoding of R, G and B and 
the reliability for the decoding is enhanced. 

(4) There is provided a spatial pattern encoding 
method of item (3) wherein blocks or areas with local 
maximum luminance values and local minimum luminance 
values of R, G and B are aligned. Since blocks with 
local maximum and minimum values are alternately 
arranged, the extraction of a matrix image (division 
of a block) can be made easier than in the prior art. 

(5) There is provided a spatial pattern encoding 
method of item (3) wherein blocks or areas with local 
maximum luminance values and local minimum luminance 
values of R, G and B are shifted. As a result, if the 
reflectances of all R, G and B signals are not at the 
noise level, the chroma become higher than in the case 
where the stripes of local maximum/minimum luminance 
values of all R, G and B are aligned, and the color 
determination process becomes easier. 

(6) There is provided a 3D image acquisition apparatus 
comprising : 

a pattern projection section which projects 
a spatial encoding pattern of item (1) ; 

a pattern memory which stores the pattern that is 
projected by the pattern projection section; 

a light reception section which receives 
reflection light; 
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an image memory which stores information of the 
light received by the light reception section; 

a depth calculation section which calculates depth 
information of an object on the basis of the pattern 
stored in the image memory and the pattern stored in 
the pattern memory; 

a 2D image information generating section which 
generates 2D image information on the basis of the 
information stored in the image memory; and 

a 3D image generating section which generates a 3D 
image on the basis of the 2D image information 
generated by the 2D image information generating 
section and the depth value calculated by the depth 
calculation section. 

As a result, since encoding for R, G and B is 
individually performed, the decoding process can be 
performed by combining the individual encoding of R, G 
and B and the reliability for the decoding is enhanced. 
(7) There is provided a 3D image acquisition apparatus 
according to item (6), comprising: 

a plurality of said light reception sections and 
a plurality of said image memories; 

a correspondency determination section which 
matches the contents of the image memories and collates 
the images; 

the depth calculation section which calculates 3D 
information of the object on the basis of information 



determined by the correspondency determination section; 
and 

a 3D image generating section which generates a 3D 
image on the basis of information generated by the 2D 
image information generating section and the value 
calculated by the depth calculation section. 

According to the structure of item (7) of the 
present invention, if the reflectances of all R, G and 
B signals are not at the noise level, the chroma become 
higher than in the case where the blocks of local 
maximum/ minimum luminance values of all R, G and B are 
aligned, and the color determination process becomes 
easier . 

As has been described above, in the present 
invention, each band of R, G and B is encoded based on 
the combination of local maximum/minimum information 
and gradation information. The range of objects for 3D 
shape measurement is not limited to white-based color 
objects and low-saturation color objects and is 
increased. 

Moreover, the present invention may provide a 3D 
image acquisition method for performing spatial pattern 
encoding and a 3D image acquisition apparatus for 
carrying out this method, wherein stripe boundary 
extraction from a stripe pattern projection image 
and block boundary extraction from a matrix pattern 
projection image can exactly be performed, and exact 
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decoding is performed from an encoded projection 
image of an object even where the object is not of 
a white-based color or a low-saturation color. 

Additional advantages and modifications will 
5 readily occur to those skilled in the art. Therefore, 

the invention in its broader aspects is not limited to 
the specific details and representative embodiments 
shown and described herein. Accordingly, various 
modifications may be made without departing from the 
10 spirit or scope of the general inventive concept as 

defined by the appended claims and their equivalents. 



